Virtual assistants for emergency dispatchers

ABSTRACT

A dispatcher virtual assistant (DVA) that can augment the capability of emergency dispatchers while reducing human errors. Major functions of the DVA include updating an emergency incident&#39;s status in real time, recommending or reminding the dispatcher to take proper actions at the right timing, answering the dispatcher&#39;s inquiries for task-related information, and fulfilling the dispatcher&#39;s request for an incident report. The DVA system includes a dispatcher language model based on machine-learning and deep-learning algorithms, for extracting the status of a live incident from incoming incident logs, and for processing and answering inquiries or requests from the dispatcher. It is customizable for different types of emergencies and for different local communities. The DVA can be used in tandem with an existing CAD system.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The subject matter disclosed herein relates generally to virtualassistants, chatbots, natural language processing, robotic processautomation, emergency dispatchers and computer-aided dispatch. Thepresent disclosure describes systems and methods of virtual assistantsfor emergency dispatchers.

2. Description of the Prior Art

Life-threatening situations, such as structure or vehicle fires, medicalemergencies, or criminal incidents, occur in any populated community ona daily basis. The reporting of such incidents is usually initiated bycitizens making emergency calls (for example, 9-1-1 calls). Call-takersin a dispatch center take these calls and relay them to a dispatcher.Based on the calls, the dispatcher gathers clues, coordinates with anddeploys first-responder units to the emergency scene. Depending on thenature of an emergency, the first responders can be firefighters,paramedics, or law-enforcement officers. Throughout the emergencyresponse process, execution and timing are both critical to the outcomeof an emergency. Mere seconds in time can result in more lives saved orlost.

Among the stakeholders of the emergency response workflow, thedispatcher clearly plays a central role. A dispatcher needs to sort outincoming calls—sometimes tens or even hundreds per incident—in realtime, assess the incident status dynamically, deploy first-responderunits most suitable for the rescuing job, and provide them withhigh-quality, critical information that allows them to form an earlyunderstanding of what they will be facing upon arrival at the emergencyscene. While in the field, first responders also need to report theironsite assessments back to the dispatcher, as well as to obtain furtherinformation from the dispatcher about the incident.

A modern dispatch center is usually equipped with a computer-aideddispatch (CAD) system (see, for example, U.S. Department of HomelandSecurity, Science and Technology, “Computer Aided Dispatch Systems”,TechNote (2011), available at:https://www.dhs.gov/sites/default/files/publications/CAD_TN_0911-508.pdf).The CAD is an integrated system for entering, sending, and loggingmessages to facilitate the emergency response workflow. Using the CAD,call-takers input information about a call, including incident location,caller identity, incident type (fire, injury, burglary, etc.), and asynopsis of the call content. A dispatcher uses the CAD to track andupdate the status of an incident, as well as to communicate with firstresponders. More advanced CAD systems are equipped with geographicinformation systems and automatic vehicle location, which help firstresponders arrive at the emergency scene faster. All messages passingthrough and information generated in the CAD system are timestamped andlogged. The logs are stored and searchable.

Such an emergency response workflow is not robust due to its criticaldependency on the skillfulness, experience and maturity of thedispatcher. Specifically, the dispatcher not only needs to sort out alarge amount of information about an incident sent from call-takers andfirst responders but also has to update the status of the incident up tothe second—often manually. Prior CAD systems can help integrate theinformation flows, but they do not solve the above problemfundamentally.

Post-incident, it is a common procedure for the dispatcher or his/hersupervisor to prepare an incident report summarizing significant factsof the entire emergency event and its handling. Sorting fragmentedincident logs into a report is a time-consuming, manpower-intensiveprocess.

The above issues are further compounded by the emergency-management jobsector having a low supply of dispatchers and a high employee turnover.Training new dispatchers is both costly and time-consuming. This can bedetrimental to the operation of any dispatch center.

Aiming at solving these problems, the present invention provides systemsand methods of virtual assistants for emergency dispatchers. Based onmachine learning and deep learning, the solutions are automated,reducing the workload and response time of dispatchers and consequently,minimizing chances for human errors.

SUMMARY OF THE INVENTION

The present invention provides dispatcher virtual assistants (DVA) thatcan augment the capability of emergency dispatchers while reducing humanerrors. Major functions of the DVA include updating an emergencyincident's status in real time, recommending or reminding the dispatcherto take proper actions at the right timing, answering the dispatcher'sinquiries for task-related information, and fulfilling the dispatcher'srequest for an incident report. The DVA system consists of a virtualassistant control unit, a dispatcher language model, an incident-statustracker, a natural language generator, a database and a graphic userinterface. It can be used in tandem with an existing CAD system.Furthermore, the DVA can be used as a cost-effective, interactivedispatcher trainer.

Based on machine-learning and deep-learning algorithms, the dispatcherlanguage model is used for extracting the status of a live incident fromincoming incident logs, and for processing and answering inquiries orrequests from the dispatcher. It is also customizable for differenttypes of emergencies and different local communities.

These and other objectives of the present invention will no doubt becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of the preferred embodiment that isillustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of the Dispatcher Virtual Assistant system(marked by the dashed box) used in tandem with a computer-aided dispatchsystem, consistent with some embodiments of the present invention. Theinformation flows within the DVA system and that between the DVA and CADsystems are indicated with solid arrows. The information flows betweenthe human stakeholders (dispatcher, call-takers and first responders)and the DVA and CAD systems are indicated with dashed arrows.

FIG. 2 is a block diagram of the architecture of the Dispatcher LanguageModel depicted in FIG. 1, consistent with some embodiments of thepresent invention.

FIG. 3 is a block diagram of the architecture of the Text Classifierdepicted in FIG. 2, consistent with some embodiments of the presentinvention.

DETAILED DESCRIPTION 1. The Dispatcher Virtual Assistant System

FIG. 1 illustrates an embodiment of the present invention. TheDispatcher Virtual Assistant (DVA) system 100 for emergency dispatcherscomprises a VA Control Unit 102, a Dispatcher Language Model 104, anIncident Status Tracker 106, a VA Database 108, a Natural LanguageGenerator 110, and a VA Graphic User Interface (VA-GUI) 112. Althoughthe DVA can be utilized as a stand-alone system, its implementation ismore efficient when used in tandem with a computer-aided dispatch (CAD)system 120, as shown in FIG. 1. The communication between the DVA andCAD systems is accomplished with an application programming interface(API). Functions of individual components of the DVA system aredescribed in the following.

The VA Control Unit 102, the heart of the DVA system, is responsible fordirecting the flows, processing, and storage of data within the DVAsystem. It receives textual data from the CAD system 120 and the VA-GUI112, and it sends the data to the Dispatcher Language Model 104 forprocessing. Depending on incident-status values it obtains from theIncident Status Tracker 106 and the inquiries or requests it receivesfrom the Dispatcher 122, the VA Control Unit 102 instructs the NaturalLanguage Generator 110 to produce either dialogue messages or incidentreports for the Dispatcher 122 to read or to dispatch. The VA ControlUnit 102 is also responsible for storing or retrieving incident data(incident logs and status) to and from the VA Database 108.

The Dispatcher Language Model 104 is responsible for processing textualdata using natural-language-processing methods involving machinelearning and deep learning algorithms. It can process at least two typesof data: incident logs received from the CAD system 120 and inquiries orrequests received via the VA-GUI 112 from the Dispatcher 122. As shownin FIG. 2, the respective methods and objectives for processing thesetwo types of data are different. From incident logs, a Text Classifier202 is used to extract text fragments (words or phrases) containing thestatus information of an emergency event. For dispatcher inquires, aNatural Language Understanding (NLU) model 204 is used to analyze theintent of each inquiry, and a Contextual Dialog Management (CDM) model206 is used to analyze the context of that inquiry. The combinedobjective of the NLU and CDM models is for the VA Control Unit 102 tounderstand the intent and context of each dispatcher inquiry and providethe best response for it via the Natural Language Generator 110. TheText Classifier 202, the NLU model 204 and the CDM model 206 are capableof processing textual data on a per-sentence basis in real time, andaccordingly, the Dispatcher Virtual Assistant 100 can provide instantservices to the Dispatcher 122.

The Incident Status Tracker 106 takes the output of the Text Classifier202 and from which deduces the status of an incident. Incident logs areunstructured textual data entered by the Call Takers 124, the FirstResponders 126 and the Dispatcher 122. To analyze the logs using theText Classifier 202, the present invention provides an incident-statusobject containing a number of properties, with each property describingone aspect of the incident. For example, the incident-status object fora building fire contains properties such as: building type(single-family house, apartments/condos, office, shopping mall, hotel,school, hospital, factory, warehouse, etc.), building construction (woodframe, joisted masonry, concrete frame, steel frame, etc.), buildingheight (number of stories), floor of fire, stage of fire (incipient,growth, fully developed, decay), color of smoke, visibility of fire,missing people, casualty information, etc. The properties are specificto incident types, and collectively, they must be able to describesufficiently the status of an incident of the particular type (fire,medical, criminal, etc.). As an emergency progresses, values of theincident-status properties are automatically extracted and updated fromincoming incident logs by the Text Classifier 202 and the IncidentStatus Tracker 106. Based on the current status of an incident, the VAControl Unit 102 provides recommendations to the Dispatcher 122 for theproper actions to take at the right timing.

The Virtual Assistant Database 108 stores timestamped incident logs andincident-status data for the VA Control Unit 102 to retrieve and use atruntime. It can also keep task-related information useful for conductingvarious dispatcher tasks. To retrieve a piece of information anytimeduring an operating or training session, the Dispatcher 122 can simplysend an inquiry to the Dispatcher Virtual Assistant 100 via the VA-GUI112.

The Natural Language Generator 110 follows the instruction of the VAControl Unit 102 to produce responses to the Dispatcher 122 in variousformats: dialogue messages, incident reports, or other forms ofinformation, depending on the use situation. To avoid any unexpectedresponse during the mission-critical dispatcher tasks, algorithms forthe Natural Language Generator 110 are preferred to be rule-based ratherthan machine-learning-based.

The Virtual Assistant GUI 112 serves as the user interface between theDispatcher 122 and the Dispatcher Virtual Assistant 100. While text andgraphics are the primary media of communication between the two, speechcommunication can be an option for the VA-GUI 112.

Hardware for implementing the Dispatcher Virtual Assistant 100 shouldinclude at least a computer server, a digital display, and standard I/Odevices such as keyboard, mouse, touch screen, microphone and speaker.

A single Dispatcher Virtual Assistant 100 may provide services formultiple dispatcher tasks in handling various types of emergency. Forfire emergency, building fires, vehicle fires and wildfires are handledas three different tasks having task-specific emergency responseworkflows. Different DVA tasks need to be separately customized.Furthermore, due to regulatory or practical considerations, a DVA mayneed to be customized for different local communities even for the samedispatcher task. The modularized architecture of the DVA system 100 ofthe present invention allows customizations to be carried out moreefficiently. For example, using the same software engines of the TextClassifier 202, the NLU 204 and the CDM 206, task-specific versions ofthe Dispatcher Language Model 104 can be prepared by training thenatural language models therein with separate task-specific data sets.

2. The Incident Status Extraction

FIG. 3 illustrates an embodiment of the Text Classifier 202 used forextracting incident-status information from incident logs. The core ofthe Text Classifier 202 is a Pre-trained Text Encoder 302, which is aTransformer-based deep-learning model of the BERT family.

The BERT model, originally developed by Google, is a text encoder thattakes a tokenized text sequence 304 (w₁ to w_(n), where n is the maximumnumber of the input text tokens) as input and provides for each token acontextualized word-embedding vector 306 as output (for the BASE versionof BERT: n=512 and the embedding-vector dimension is 768; see J. Devlin,M. W. Chang, K. Lee, K. Toutanova, “BERT: Pre-training of DeepBidirectional Transformers for Language Understanding”, arXiv:1810.04805(2019)). The BERT model has been pre-trained on a large corpus of textcontaining a vocabulary of some 30,000 text tokens (words andsub-words), and hence it is used as a generic text encoder. Aimed atimproving the original BERT model, a number of BERT derivatives havebeen developed and pre-trained (see, for example, the review article byA. Rogers, O. Kovaleva, A. Rumshisky, “A Primer in BERTology: What WeKnow About How BERT Works”, arXiv:2002.12327 (2020)). ELECTRA is a morerecent addition to the BERT family that has seen a great deal of successin natural language processing (K. Clark, M. T. Luong, Q. V. Le, C. D.Manning, “ELECTRA: Pre-training Text Encoders as Discriminators RatherThan Generators”, arXiv:2003.10555 (2020)).

The decoder of the Text Classifier 202 comprises a Fully Connected Layer308 and a Softmax operation 310. The output of the Text Classifier 202comprises multi-class text tokens 312. The output text tokens are wordsor phrases relevant to the incident status extracted from the input textsequence 304 (any sentence of the incoming incident logs).

At runtime, the Text Classifier 202 assigns each output text token aclass corresponding to a property of the incident-status object. Forexample, for an input message: “I saw black smoke gushing out of thetenth-floor windows of Hotel XYZ.”, the Text Classifier 202 will outputthree text tokens: “black smoke”, “tenth-floor” and “Hotel XYZ”,respectively labeled as “color of smoke”, “floor of fire” and “buildingtype”. The values: “black”, “10” and “hotel”, for the correspondingproperties of the incident-status object will then be extracted by theIncident Status Tracker 106.

Before the Dispatcher Virtual Assistant 100 can be used for a specifictask, the Dispatcher Language Model 104 needs to be trained, and boththe Incident Status Tracker 106 and the Natural Language Generator 110need to be customized for that task. The training data for a specificdispatcher task can be obtained from historical logs of many incidentsfor that task. These data need to be annotated for training.Empirically, for an incident-status object containing 30 properties, anannotated training corpus of a few thousand log messages are sufficientto train an effective Dispatcher Virtual Assistant of the presentinvention.

3. The Dispatcher Virtual Assistant Process Cycles

Atypical emergency response process is triggered by a citizen making thefirst call about an emergency incident to a dispatch center. From thatmoment to the resolution or closure of the incident, there could be tensor even hundreds of calls and text messages concerning the incident,either between citizens (not shown in FIG. 1) and the Call Takers 124,or between the First Responders 126 and the Dispatcher 122, as depictedin FIG. 1. These communications are logged within the CAD system 120.

The role of the Dispatcher Virtual Assistant 100 is to assist theDispatcher 122 to carry out his/her existing workflows with elevatedefficiency and reduced chances of human errors. A great advantage of thepresent invention is that the DVA can be incorporated into an existingemergency response process without changing the original workflow. Thatis, the DVA can carry out its own process cycles in parallel with themain workflow.

Major functions of the DVA include (A) updating the incident status inreal time, (B) recommending or reminding the dispatcher to take the nextaction at the right timing, (C) answering the dispatcher's inquiries fortask-related information, and (D) fulfilling the dispatcher's requestfor an incident report. These functions are respectively implementedwith four corresponding process cycles. The process cycles for functions(A) and (B) take place autonomously during any active emergencyincident, and those for functions (C) and (D) are reactively triggeredby the dispatcher. The four process cycles are described in thefollowing passages.

A. Updating the incident status in real time:

(1) Referring to FIG. 1, during an active emergency incident, a new logmessage is entered into the CAD system 120 by a Call Taker 124 or aFirst Responder 126.(2) The message is forwarded to the VA Control Unit 102 via anapplication programming interface (API).(3) The VA Control Unit 102 stores the log message to the VA Database108 and concurrently sends the message to the Dispatcher Language Model104 for analysis.(4) From the log message, the Dispatcher Language Model 104 extractstext tokens (words or phrases) relevant to the incident status and sendsthe result to the Incident Status Tracker 106.(5) From the text tokens of step (4), the Incident Status Tracker 106deduces the corresponding property values for the incident-statusobject.(6) The Incident Status Tracker 106 sends the updated incident-statusobject to the VA Control Unit 102, and the latter stores the data to theVA Database 108.

The information flow for this process cycle is depicted sequentiallywith the arrows 130/132, 134, 136 & 138, 140, 142, and 144.

B. Recommending the dispatcher for the next action:

(1) During an active emergency incident, if the VA Control Unit 102judges from the current incident status that the Dispatcher 122 shouldtake a new action or complete a pending one, it sends an instruction tothe Natural Language Generator 110 for recommending or reminding theDispatcher 122 to take the proper action.(2) According to the instruction of step (1), the Natural LanguageGenerator 110 sends a recommendation or reminder message to the VA-GUI112.(3) Via the VA-GUI 112, the Dispatcher 122 receives the recommendationor reminder from the DVA 100 for taking his/her next action.(4) If the aforementioned action involves communicating with the FirstResponders 126, it is carried out by the Dispatcher 122 with the help ofthe CAD 120.

The information flow for this process cycle is depicted sequentiallywith the arrows 146, 148, 150, 152, and 154.

C. Answering the dispatcher's inquiries for task-related information:

(1) During an active emergency incident or a training session, theDispatcher 122 can ask for task-related information from the DVA 100.Such an inquiry is entered via the VA-GUI 112.(2) The VA-GUI 112 forwards the inquiry to the VA Control Unit 102.(3) The VA Control Unit 102 sends the inquiry to the Dispatcher LanguageModel 104 for analysis.(4) Having analyzed the intent and context of the dispatcher's inquiry,the Dispatcher Language Model 104 sends a direction to the NaturalLanguage Generator 110 to provide a proper response to the Dispatcher122.(5) The Natural Language Generator 110 sends the response to theDispatcher 122 via the VA-GUI 112.

The information flow for this process cycle is depicted sequentiallywith the arrows 160, 162, 164, 166, 168, and 170.

D. Fulfilling the dispatcher's request for an incident report:

(1) At any time during or after an emergency incident, the Dispatcher122 can request the DVA 100 to issue an incident report. Such a requestis entered via the VA-GUI 112.(2) The VA-GUI 112 forwards the request to the VA Control Unit 102.(3) The VA Control Unit 102 retrieves the latest version of theincident-status object from the VA Database 108 and sends it to theNatural Language Generator 110.(4) Using the incident-status data received in step (3), the NaturalLanguage Generator 110 produces an incident report and sends it to theCAD 120 for further dispatching.

The information flow for this process cycle is depicted sequentiallywith the arrows 180, 182, 184, 186, 188, and 190.

In summary, the present invention provides dispatcher virtual assistants(DVA) that can augment the capability of emergency dispatchers whilereducing human errors. Major functions of the DVA include updating anemergency incident's status in real time, recommending or reminding thedispatcher to take proper actions at the right timing, answering thedispatcher's inquiries for task-related information, and fulfilling thedispatcher's request for an incident report. The DVA system includes avirtual assistant control unit, a dispatcher language model, anincident-status tracker, a natural language generator, a database and agraphic user interface. It can be used in tandem with an existing CADsystem. Furthermore, the DVA can be used as a cost-effective,interactive dispatcher trainer.

Based on machine-learning and deep-learning algorithms, the dispatcherlanguage model is used for extracting the status of a live incident fromincoming incident logs, and for processing and answering inquiries orrequests from the dispatcher. It is customizable for different types ofemergencies and for different local communities.

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the invention. Accordingly, the abovedisclosure should be construed as limited only by the metes and boundsof the appended claims.

What is claimed is:
 1. A virtual assistant (VA) system for augmenting emergency dispatchers' capability in handling live emergency incidents, comprising: a VA control unit for making decisions and directing information flows, either based on incoming incident logs of a said incident, or according to inquiries or requests from a said dispatcher; a dispatcher language model, coupled to the VA control unit, for extracting, in real time, incident-status-related text fragments from said incident logs, and for analyzing inquiries or requests from said dispatcher; an incident state tracker, coupled to the VA control unit, for deducing, in real time, an incident-status object comprising a set of properties that describe aspects of said incident from said incident-status-related text fragments; a VA database, coupled to the VA control unit, for storing dispatcher-task-related information, said incident logs and property values of said incident-status object; a natural language generator, coupled to the VA control unit, for generating text messages or incident reports, either autonomously based on property values of said incident-status object, or reactively according to inquiries or requests from said dispatcher; and a VA graphic user interface, coupled to the VA control unit, for said dispatcher to communicate with said VA system.
 2. The dispatcher virtual assistant system of claim 1, wherein said dispatcher language model comprises: a text classifier for extracting said incident-status-related text fragments from said incident logs; a natural language understanding model for analyzing intents of inquiries or requests received from said dispatcher; and a contextual dialog management model for analyzing contexts of inquiries or requests received from said dispatcher.
 3. The dispatcher virtual assistant system of claim 2, wherein said text classifier comprises: a pre-trained text encoder of the Bidirectional Encoder Representations from Transformers (BERT) family; and a decoder comprising a fully connected layer and a Softmax operation; wherein said text classifier taking said incident logs, sentence by sentence, as input and extracting text fragments, labeled with properties of said incident-status object, as output.
 4. Methods of the dispatcher virtual assistant system of claim 1, comprising combinations of the following process cycles: autonomously extracting updated property values of said incident-status object from said incident logs in real time; autonomously recommending or reminding said dispatcher to take proper actions at the right timing based on updated property values of said incident-status object; processing and answering said dispatcher's inquiries for dispatcher-task-related information; and generating incident reports according to said dispatcher's requests. 